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[57] ABSTRACT 

A method for altering a linked series of image processors 
capable of manipulation digital image data includes provid- 
ing a digital image capture device with an image processing 
chain, and modifying the image processing chain to form a 
new image processing chain. The modifying further includes 
deleting an image processor from the image processing 
chain. Alternatively, the modifying further includes altering 
the image processing chain with at least one externally 
provided image processor, inserting an image processor into 
the image processing chain, or replacing an image processor 
in the image processing chain. 

In a system aspect, the system includes a digital image 
capture device, the digital image capture device capable of 
processing digital image data through two or more image 
processors. The system further includes a central processing 
unit within the digital image capture device and capable of 
linking the two or more image processors to form an image 
processing chain. The central processing unit further facili- 
tates modification of the image processing chain to form a 
new image processing chain. 

17 Claims, 8 Drawing Sheets 
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FIG. 1 
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FLEXIBLE DIGITAL IMAGE PROCESSING processing digital image data through two or more image 

VIA AN IMAGE PROCESSING CHAIN WITH processors. The system further includes a central processing 

MODULAR IMAGE PROCESSORS unit within the digital image capture device and capable of 

linking the two or more image processors to form an image 
RELATED APPLICATIONS 5 processing chain. The central processing unit further facili- 

The present invention is related to co-pending U.S. patent »atcs modification of the image processing chain to form a 
application, Ser. No. 08/705,619, filed on Aug. 29, 1996, new processing chain. 

now U.S. Pat. No. 6,028,611 entitled MODULAR DIGITAL With the present invention, processing of digital image 
IMAGE PROCESSING VIA AN IMAGE PROCESSING data occurs with a linked series of image processors. Each 
CHAIN, and assigned to the assignee of the present inven- 10 of the image processors performs some level of manipula- 
tion, tion of the digital image data. The separation of digital image 

The present invention is also related to co-pending U.S. processing into a series of image processors allows a more 
patent application, Ser. No. 08/705,325, filed on Aug. 29, modular approach to processing digital image data. Further, 
1996, now U.S. Pat. No. 5,991,465, entitled MODULAR the present invention uniquely allows modification of the 
DIGITAL IMAGE PROCESSING VIA AN IMAGE PRO- 15 series through deletion of an image processor, insertion of a 
CESSING CHAIN WITH MODIFIABLE PARAMETER different image processor, or replacement of an existing 
CONTROLS, and assigned to the assignee of the present image processor. In addition, aspects of an image processor, 
invention. including parameter control values, are alterable in accor- 

dance with a preferred embodiment to allow greater adapt - 
FIELD OF THE INVENTION 20 ability to user-specific design preferences. Enhancements 

The present invention relates to digital image data <* a ng es 10 the chain «* therefore easily achieved, 

processing, and more particularly to flexible digital image allowin g flexibility and more convenient upgrading 

data processing with modular image processors. of dl gi tal una g e processing. 

These and other advantages of the aspects of the present 
BACKGROUND OF THE INVENTION 25 ^v^on be more fully understood in conjunction with 

Modern digital cameras typically include an imaging ihe following detailed description and accompanying draw- 
device which is controlled by a computer system. The in&s- 

computer system accesses raw image data captured by the nPSPRTPTION OF THE DRAWINGS 

imaging device and then processes and compresses the data 3Q BRIEF DESCRIPTION OF THE DRAWINGS 

before storing the compressed data into an internal memory. piG 1 ^ a block dj a g ram 0 f a digital camera that operates 
The conventional digital camera captures raw image data lQ accorc jance with the present invention, 
and then remains unusable until the data is completely FIG. 2 is a block diagram of the preferred embodiment for 
processed and stored into internal flash memory. ^ imaging deyice of piG L 

In processing image data, typical digital cameras operate 35 nG ^ Qf me ^ cmbodimenl for 

with exclusive and specific image processing. Thus, all the ^ computer of FIG 1 

potential manipulations on image data, such as linearization, p * ... . 

sharpening, and compression, occur as a result of isolated ™. 4 is a memory map showing the preferred embodi- 
prcsct programming and/or specifically designed hardware. ment of the read only memory (ROM) of FIG. 3. 

While some level of manipulation of image data is 40 HG. 5 is a block diagram showing preferred data paths for 
achieved with the programming or hardware, attempts to transmitting image data between components of the FIG. 3 
alter and improve the processing are hampered by the rigid computer. 

structure of using a single file/specific components. FIG. 6 illustrates an image processing chain of three 
Furthermore, camera functionality remains tied to technol- image processors. 

ogy available at the time of the design and is not readily 45 FIG. 7 illustrates a more specific example of the image 
replaced and updated as technology improves. Accordingly, processing chain. 

a need exists for a more flexible, modular approach for FIG 8 flhislralcs an j^ge processing backplane in con- 
processing digital image data that provides enhanced digital junction ^ib the image processing chain of FIG. 6. 
image output through an adaptable image processing sys- 
tem. 50 DETAILED DESCRIPTION 

SUMMARY OF THE INVENTION ^ present i nve ntion relates to a flexible, modular 

Accordingly, the present invention meets these needs and approach to processing of digital image data. The following 

provides a method and system for altering a linked series of description is presented to enable one of ordinary skill in the 

image processors capable of manipulating, digital image 55 art t0 make and ^ invention and is provided in the 

data. In a method aspect, the method includes providing a context of a patent application and its requirements. Various 

digital image capture device with an image processing modifications to the preferred embodiment and the generic 

chain, and modifying the image processing chain to form a principles and features described herein will be readily 

new image processing chain. The modifying further includes apparent to those skilled in the art. 

deleting an image processor from the image processing 60 Although the following describes processing of digital 

chain. Alternatively, the modifying further includes altering image data captured through a digital camera device, it is 

the image processing chain with at least one externally meant as an illustrative embodiment of the features of the 

provided image processor, inserting an image processor into present invention. The present invention is equally capable 

the image processing chain, or replacing an image processor of utilization with other devices that perform digital image 

in the image processing chain. $s data capture and processing, including, but not limited to, 

In a system aspect, the system includes a digital image computer systems, including those used to capture digital 

capture device, the digital image capture device capable of images accessible from Internet sites and image scanner 
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equipment. Further, the data structures and commands dis- of camera 110. In the preferred embodiment, CPU 344 is 

cussed with reference to a preferred embodiment are suit- capable of concurrently running multiple software routines 

ably included as part of high level code used directly by one to control the various processes of camera 110 within a 

or more applications that is readily achieved through the use multi-threading environment. DRAM 346 is a contiguous 

of C, C++, or other., similar programming language, and 5 block of dynamic memory which may be selectively allo- 

stored on a computer readable medium. cated to various storage functions. 

A digital camera architecture has been disclosed in I/O 348 is an interface device allowing communications 

co-pending U.S. patent application Ser. No. 08/666,241, to and from computer 118. For example, I/O 348 permits an 

entitled "A System And Method For Using A Unified external host computer (not shown) to connect to and 

Memory Architecture To Implement A Digital Camera 10 communicate with computer 118. I/O 348 also permits a 

Device," filed on Jun. 20, 1996, and assigned to the Assignee camera U0 user to communicate with camera 110 via an 

of the present application. The Applicant hereby incorpo- external user interface and via an external display panel, 

rates the co-pending application by reference, and repro- referred to as a view finder. 

duces portions; of that application herein with reference to ROM 350 typically comprises a conventional nonvolatile 

FIGS. 1-5 for convenience. 15 read-only memory which stores a set of computer-readable 

Referring now to FIG. 1, a block diagram of a camera 110 program instructions to control the operation of camera 110. 

is shown according; to the present invention. Camera 110 ROM 350 is further discussed below in conjunction with 

preferably comprises an imaging device 114, a system bus FIG. 4. Removable memory 354 serves as an additional 

116 and a computer 118. Imaging device 114 is optically image data storage area and is preferably a non-volatile 

coupled to an object 112 and electrically coupled via system 20 device, readily removable and replaceable by a camera 110 

bus 116 to computer 118. Once a photographer has focused user via buffers/connector 352. Thus, a user who possesses 

imaging device 114 on object 112 and, using a capture several removable memories 354 may replace a full remov- 

button or some other means, instructed camera 110 to able memory 354 with an empty removable memory 354 to 

capture an image of object 112, computer 118 commands effectively expand the picture-taking capacity of camera 

imaging device 114 via system bus 116 to capture raw image 25 HO. In the preferred embodiment of the present invention, 

data representing object U2. The captured raw image data is removable memory 354 is typically implemented using a 

transferred over system bus 116 to computer 118 which flash disk. 

' performs various image processing functions on the image Power supply 356 supplies operating power to the various 

data before storing it in its internal memory. System bus 116 components of camera HO. In the preferred embodiment, 

also passes various status and control signals between imag- 30 power supply 356 provides operating power to a main power 

ing device 114 and computer 118. bus 362 and also to a secondary power bus 364. The main 

Referring now to FIG. 2, a block diagram of the preferred power bus 362 provides power to imaging device 114, I/O 

embodiment of imaging device U4 is shown. Imaging 348, ROM 350 and removable memory 354. The secondary 

device 114 preferably comprises a lens 220 having an iris, a 35 power bus 364 provides power to power manager 342, CPU 

filter 222, an image sensor 224, a timing generator 226, an 344 and DRAM 346. 

analog signal processor (ASP) 228, an analog-to-digital Power supply 356 is connected to main batteries 358 and 

(A/D) converter 230, an interface 232, and one or more also to backup batteries 360. In the preferred embodiment, 

motors 234. a camera 110 user may also connect power supply 356 to an 

U.S. patent application Ser. No. 08/355,031, entitled "A ^ external power source. During normal operation of power 

System and Method For Generating a Contrast Overlay as a supply 356, the main batteries 358 provide operating power 

Focus Assist for an Imaging Device," filed on Dec. 13,1994, to power supply 356 which then provides the operating 

is incorporated herein by reference and provides a detailed power to camera HO via both main power bus 362 and 

discussion of the preferred elements of imaging device 114. secondary power bus 364. 

Briefly, imaging device H4 captures an image of object 112 45 During a power failure mode in which the main batteries 

via reflected light impacting image sensor 224 along optical 358 have failed (when their output voltage has fallen below 

path 236. Image sensor 224 responsively generates a set of a minimum operational voltage level) the backup batteries 

raw image data representing the captured image 112. The 360 provide operating power to power supply 356 which 

raw image data is then routed through ASP 228, A/D then provides the operating power only to the secondary 

converter 230 and interface 232. Interface 232 has outputs 50 power bus 364 of camera HO. Selected components of 

for controlling ASP 228, motors 234 and timing generator camera 110 (including DRAM 346) are thus protected 

226. From interface 232, the raw image data passes over against a power failure in main batteries 358. 

system bus 116 to computer H8. p 0W er supply 356 preferably also includes a flywheel 

Referring now to FIG. 3, a block diagram of the preferred capacitor connected to the power line coming from the main 

embodiment for computer U8 is shown. System bus 116 55 batteries 358. If the main batteries 358 suddenly fail, the 

provides connection paths between imaging device H4, flywheel capacitor temporarily maintains the voltage from 

power manager 342, central processing unit (CPU) 344, the main batteries 358 at a sufficient level, so that computer 

dynamic random-access memory (DRAM) 346, input/ 118 can protect any image data currently being processed by 

output interface (I/O) 348, read-only memory (ROM) 350, camera U0 before shutdown occurs, 

and buffers/connector 352. Removable memory 354 con- 50 Referring now to FIG. 4, a memory map showing the 

nects to system bus 116 via buffers/connector 352. preferred embodiment of ROM 350 is shown. In the pre- 

Alteraately, camera 110 may be implemented without ferred embodiment, ROM 350 includes control application 

removable memory 354 or buffers/connector 352. 400, toolbox 402, drivers 404, kernal 406 and system 

Power manager 342 communicates via line 366 with configuration 408. Control application 400 comprises pro- 
power supply 356 and coordinates power management 65 gram instructions for controlling and coordinating the van- 
operations for camera HO. CPU 344 typically includes a ous functions of camera 110. Toolbox 402 contains selected 
conventional processor device for controlling the operation function modules including memory manager 410, RAM 
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spooler 1 (412), RAM spooler 2 (414), removable memory By way of example, linearization refers to straightforward 

spooler 1 (416), removable memory spooler 2 (418), image conversion of the image data from an eight-bit non-linear 

processing and compression 420 and file system 422. space to sixteen-bit linear space. As a more specific 

Referring now to FIG. 5, a block diagram showing example, input pixels stored as eight bit compressed Bayer 

preferred data paths for transmitting image data between 5 pattern image data are converted through linearization image 

selected computer 118 components is shown. In FIG. 5, processor 502a into sixteen bit extended Bayer pattern 

frame buffer 536 receives and stores raw image data previ- image data. 

ously captured by, image device 114. Frame buffer 536 then A nexl suitable image processor 5026 is a bad pixel 

transfers control of the raw image data to RAM spooler 1 replacement processor. Bad pixel replacement suitably 

(412) via line 610. Alternatively, if RAM disk 532 is full, 10 occurs throu S h interpolation of the neighborhood pixels 

frame buffer 536 may transfer control of the raw image data arouod the defective CCD pixels. Tne processing by image 

directly to image processing/compression 420 using line P roccss ? r 5 , 02 * capably receives and outputs pixel data in 

612. If RAM spooler 1 (412) receives control of the raw slx f °- blt lmear s P ace Bayer format 

image data, it then stores the raw image data into RAM disk c ^s a next image processor 502, white balance processor 

532 ' 1" e 614 performs white balance image processing. Pixel data 

using me 15 rece j vec j aD( j output by the white balance image processor 

Removable memory spooler 1 (416) may then access the 5Q2c m appropriately stored in sixteen-bit linear space 

raw image data from RAM disk 532 via line 616 and store Bayer format 

it into removable memory 354 using line 618. Alternatively Afoimh [m ^ 50 2rf preferably performs image 

if removable memory 354 is frill or is not inserted, RAM colof Qr ^ fiUw ^ (a?A) reconstructiorh B 

disk 532 may provide the raw image data dnectly to image 20 Qf ex ^ CFA reconslniction ^ ^ 5Q2d 

processing/compression 420 using line 620. If removable suitabl / achieves an interpolation operation to convert 

memory spooler 1 (416) stores the raw image data into sixteeQ . bil B CFA uern CCD data ^ a forty ^ ight 

removable memory 354, then image processing/ bit extended RGB image. 

compression 420 typically accesses the stored raw image n . . M , 

« . Following CFA reconstruction image processor 502d, 

data using line 622. 25 _ 6 * r . . , 1 

& color transformation image processor 502e is included. An 

LINKING IMAGE PROCESSORS FOR appropriate color transformation image processor 502e 

FORMING IMAGES employs a color correction matrix, such as to convert from 

In the preferred embodiment, image processing and com- device-dependent camera color space to device-independent 
pression 420 occurs via an image processing chain (IPC). 30 linear CCIR709 color space. Preferably, the input and output 
For purposes of this discussion, the IPC preferably refers to P ixel data is slored m forty-eight bit extended RGB format, 
a software process that manipulates image data in a stage by As a next image processor, YCC color space transforma- 
stage fashion. As shown in FIG. 6, an IPC 500 is suitably tion image processor 502/ is included. The YCC color space 
composed of a sequence of image processors 502 with each conversion image processor 502/ suitably uses CCIR 601-2 
image processor 502 performing a particular type of image 35 specification to create an eight-bit YCrCb image from an 
transformation. The input image data 504 is suitably RGB image. Input pixel data to image processor 502/ is 
received from a single image source and output as output suitably given in forty-eight bit extended RGB format with 
image data 506 into a single image destination. Image output pixel data in twenty-four bit YCrCb444 format, 
processors suitably refer to software modules that apply Two additional image processors 502 in the IPC 500 
algorithms on image data to obtain a special image process- 4 q include sharpening image processor 502g and JPEG com- 
ing result, specific examples of which are described below pression image processor 502/i. Sharpening image processor 
with reference to FIG. 7. 502g suitably receives input pixel data in twenty- four bit 

FIG. 7 illustrates the IPC 500 with several examples of the YCrCb format and outputs pixel data in the same format 

image processors 502 capable for utilization as the IPC 500. after performing sharpening operations. Parameter control 

For each of the image processors 502, an unambiguous 45 of the sharpening suitably occurs with a range of values for 

image data format is specified for the input and output data. the sharpening operation. 

When the input and output image data formats are the same, The JPEG compression image processor 502^ suitably 
the image processor 502 is considered non-transforming, performs JFIF base line image compression. Input pixel data 
examples of which are represented by the rounded boxes in in twenty-four bit YCrCb444 format is output from image 
FIG. 7. Conversely, image processors 502 that do not have 50 processor 502A as compressed and subsampled YCC format, 
the same input and output data formats are suitably consid- 48-bit YCrCb411 per 4 pixel data. Two forms of parameter 
ered transforming, e.g., the rectangles of FIG. 7. Although controls are achieved via image processor 502/i to both 
the following description of FIG. 7 is given with a particular control the degree of compression, e.g., maximum to normal 
order and series of image processors for image processing to to lossless, and to identify data as color or grayscale, 
occur in a sequential and serial manner, it should be appre- 55 Coordination of the image processors 502 to form the IPC 
ciated that in the preferred embodiment, any number of 500 is preferably done via an image processing backplane 
non-transforming image processors may be chained between (IPB). In a preferred embodiment, the image processing 
two separate transforming image processors. Further, brief backplane provides processing support in a broad manner to 
descriptions of the type of image processing capable by each allow varying algorithms to be incorporated as image pro- 
image processor 502 are included as examples. However, the 60 cessors 502. The features of the processing support by the 
details of such processing are not included in the present IPB are described in more detail with reference to FIG. 8 and 
discussion and are considered to be well understood by those include performing image scan line buffer input/output 
skilled in the art. Thus, image processing through the use of (I/O), IPC construction and connection, image processor 
other image processors in the IPC 500 is within the spirit and parameter control setting, single pass through image data, 
scope of the present invention. 65 procedural interface to the image processors, circular data 
The image processors 502 suitably include a first image pipeline support, and ring-pixel handling, with minimal 
processor 502a for linearization of the input image data 504. memory requirements and overheads. 
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FIG. 8 illustrates schematically an IPB 520 in conjunction processor 502 into the IPC 500. Preferably, the installation 

with an IPC 500 including two image processors 502. function specifies an IPC reference number, as identified in 

Suitably, internal data structures, e.g., IPBbmageBuf 522 the initialization function, and provides pointers to the seven 

and IPBNode 524, are maintained by the IPB 520 for storing functional routine entries, as discussed hereinbelow, of the 

information related to the image processors 502 during 5 image processor being installed. 

processing and are connected indirectly in terms of data ^ ^ oonncction fsxMioa mes m IPC refcrence 

flow, as indicated by the dashed arrows in FIG. * The 520 aU cessors 

mformation mamtained by the data structures 522 and 524 , , ? , n . . t . . # . CAn & / . ... 

preferably includes locations of the input and output line 502 ha \ ebeea inslallcd 500 000131,15 aU ,he 

buffers, aid internal state and functional routine enuy point- „ 2*"%*™* *?*^* *° 2 to perform 

c . . crt - ^, . . mo 10 The IPC destruction function specifies an IPC reference 

ers of each image processor 502. Thus, data structure 522 . . caa a i.u u 

. . . • • . • ♦ • * ■ i «■ number and is called to destroy an IPC 500. Although a 

capably contains pointers pointing to image scan line buffers mp iAft . f , . . . # . ~. 

-i ,. , • * . f t - • default camera IPC 500 is unlikely to be destroyed, other 

526 that are used to store input and output image data in _ 7 u r 

r . ui j . • j / • • . ■ IPCs added to a camera for other purposes, by functions in 

formats suitably determined during installation of an image F *~ * f 

• i i .f • ,. . rr accordance with a preferred embodiment and discussed in 

processor 502. More particularly, the image scan line buffers ^u«uww « « u ...... 

r ui . • - . _ ■ 15 more detail herembelow, are suitably destroyed with this 

526 preferably store one or more image scan lines, i.e., the . ' J J 

lines of data forming a data pipeline that consists of the DC 100 ' 

minimum number of lines required by an image processor MODULARITY OF IMAGE PROCESSORS 

502 THROUGH FLEXIBLE UPDATING OF AN IPC 

For image processors 502, processing suitably occurs 2 q 

with a data pipeline that contains a single image scan line, Alterations to an existing IPC 500 readily occurs in a 

i.e., an image pixel line in the fast scan direction from left preferred embodiment through an update function, e.g., 

to right. However, some image processors 502, such as IPBUpdateDefaultlPC, that specifies the IPC reference num- 

compression image processors (e.g., 502A, FIG. 7), utilize ber for the IPC 500 being updated/modified. Updating of an 

more than one scan line during processing to take neigh- 25 IPC 500 includes insertion of an image processor 502 to the 

boring effects into account. When more than one scan line is IPC 500, deletion of an image processor 502 from the IPC 

needed by an image processor 502, a data pipeline is suitably 500, or replacement of an image processor 502 with an 

defined for convenience at the input end of the image alternate image processor 502. Preferably, the default IPC 

processor 502. For purposes of this discussion, a data 500 is updated via an image processor module on a storage 

pipeline refers to a minimum collection of image scan lines 30 device, e.g., removable memory, RAM disk, or internal 

required by an image processor 502. Generally, a data memory. The image processor module suitably contains one 

pipeline includes an image scan line currently being or more plug-in image processors that each have one addi- 

processed, and some number of image scan lines prior to tional function, e.g., IPMPluglnProc, that defines the updat- 

('lookback') and/or after ('lookahead') the current image ing strategy, the signature of the target image processor to be 

scan line. Suitably, access to the data pipeline occurs via a 35 updated, and pointers of the seven basic functions of an 

circular array of buffer pointers, so that after each processing image processor, as described hereinbelow. Lack of identi- 

iteration of the image processor 502, the pointers in the array fication of valid target image processors in an IPC or lack of 

are circularly rotated, as is well understood by those skilled match between the format of the output of one image 

in the art. In contrast to prior devices that typically require processor and input of a next image processor chained 

large amounts of memory to perform image data ^ together preferably results in cancellation of the updating 

manipulations, the image scan line buffers provide suflScient attempt and restoration of the default IPC 500. 

memory to perform processing one scan line at a time, thus Defining an image processor 502, for use in a default IPC 

reducing the overall memory requirements without reducing 500 or as an updating image processor, suitably occurs 

processing capabilities. through seven functional routines or procedures, as indi- 

Preferably, the data pipeline required by an image pro- 45 cated by block 528 in FIG. 8. A definition function, e.g., 

ccssor 502 is indicated during the installation of the image IPDefineProc, allows an image processor 502 to specify its 

processor 502 in the IPC 500. Installation of an image characteristics. It is appropriately called by the IPB 520 

processor 502 suitably occurs when the camera first starts up when the image processor 502 is installed into the IPC 500 

with an IPC 500 constructed from all of the default image to identify the characteristics of the image processor 502. By 

processors 502 stored in the system ROM. Suitable func- 50 way of example, for an image processor 502 that performs 

tions to coordinate the construction and deconstruction of color correction via a 3x3 matrix, input and output formats 

the IPC 500 include four functions, an initialization of 48 bit extended RGB are capably identified by the 

function, e.g., IPCInit, an installation function, e.g., definition function. Further characteristics identified include 

IPCInstalllmageProcessor, a connection function, e.g., the configuration of the data pipeline associated with the 

IPCConnect, and a destruction function, e.g., IPCDeslroy. 55 image processor 502, the number of ring-pixels, and the 

The IPC initialization function is called to create a new number of parameter controls. 

IPC 500. A suitable default IPC 500 converts raw CCD An initialization function, e.g., IPInitProc, appropriately 

capture data into a JPEG compressed image. Preferably, the allows an image processor 502 to allocate any internal 

IPC initialization function returns a reference to a new storage it might need when processing an image. It is 

image processing chain, identifies types of image processors 60 suitably called by the IPB 520 only once when the image 

included in the chain, and specifies a maximum expected processor 502 is installed into the IPC 500. Subsequent calls 

width in pixels to be sent through the IPC, where the to the other five functions described below then pass the 

maximum width includes ring-pixels, which refer to supple- internal storage space allocated by the IPInitProc as an 

mentary image data at each side of the image required by an argument. Further identified by the initialization function is 

image processor to perform a particular algorithm. 65 the maximum width specification of an image scan line in 

The IPC installation function is called by an image pixels that is expected at the input, including ring-pixels at 

processing application to the IPB 520 to install the image both the left and right sides. In the example of defining the 
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color correction image processor, the initialization function an external mechanism, such as a control application 400 

capably identifies a memory location storing a pointer to (FIG. 4). These functions include parameter control capa- 

needed 3x3 matrix constant value internal variables, and the bility and value determination functions, e.g., 

maximum width. IPBGetParameterCapability, IPBGetDefaultParameter, and 

Two functions, a control function, e.g., IPControlProc, 5 IPBGetParameter. Also included are functions for setting or 

and a status function, e.g., IPStatusProc, deal with parameter restoring parameter control values, e.g., 

controls of an image processor 502. Preferably, parameter IPBSetDefaultParameter, I PBSet Parameter, and IPBRe- 

controls for an image processor 502 each have a unique storeParameter. Preferably, for the parameter control value 

4-character tag that is registered to avoid conflict. Parameter determination functions, an IPC 500 and the number of 

control values include two types, a range type and an iQ parameters requested are identified, as weU as identificaUon 

enumerated list type Range types of parameter control of a mtef tQ aQ of arameter ta ^ a ^ inter t0 a 

values are appropriately ^confined tetween the minimum and ffie locatkm ^ J ^ imer % ( meter 

maximum settings for the range. Enumerated list parameter „■ . a a • . . 1 u 

control values assign different enumerated numbers to dif- relumed, and a pointer to a memory location where 

ferent settings with a 32-character null terminated string * he n , umbe ' of ^ les of parameter control values are stored 

used to provide an ASCII name for each enumerated list 15 Similarly for the capability determination function, an IPC 

number. Examples of parameter controls include sharpening 15 specified, the number of parameters requested is specified, 

values (range type), color specification control values (range a pointer to an array of parameter tags is specified, a pointer 

type), and compression control values (enumerated list t0 a memory location used to store a pointer for the param- 

type). eter capability information returned is specified, and a 

The control function, IPControlProc, is called by the IPB 20 pointer to a memory location where the number of bytes of 

520 to control the processing parameters one parameter parameter capability information is stored is specified. Thus, 

control at a time and only before a reset function, e.g., access to the parameter controls managed by an IPC 500 are 

IPReset, call for every image to be processed. The status available, as well as current values, device dependent fac- 

function, IPStatusProc, allows an image processor 502 tory default values, and user-specified default values, 

called by the IPB 520 to determine any parameter kind, 25 For the parameter control value setting functions, prefer- 

values types, factory default parameter setting, and current ably identified by the functions are an IPC, a number of 

parameter setting of an image processor 502. No-operation parameter control values to be set, and a pointer to a list of 

routines are provided when the image processor does not parameler lags and either a current value or a user default 

support any parameter settings, such as in the example of the value ^ afe {Q be M b ^ fanction , Q me M 

color correction image processor. 30 etef fonctioD appropr i a tely allows all parameters not 

Tne reset function, I PResetProc suitably allows IPB 520 Usted {q ^ ^ parame ter value list to be reset to their 
to signal an image processor 502 to reset any internal ^ ^ ^ BooleaQ ^ Tfae 
variables used by the image processor 502 before every • 1 1 -j mr> a u 
image is processed With the color correction image proces- restoration function similarly identifies an IPC and a number 
sor example, no operation routines are provided, since no „ of parameters to be requested, provides a pointer to ao array 
local variables need to be reset. A process function, e.g., 35 of parameter tags, and selects the type of parameter defaults, 
IPProcessProc, suitably allows an image processor 502 to i e > user-specified or device dependent factory, being reset 
process image data one scan line at a time. It is suitably through a Boolean variable. These functions therefore pro- 
called whenever a data pipeline for the image processor 502 vide convenient accessibility to allow alteration of param- 
fills up. Thus, the operations for performing the 3x3 matrix eter control values in an IPC 500. Greater flexibility for 
manipulation in the color correction image processor 40 adjusting an image processor 502 within an IPC 500 is 
example, are specified with the process function. A destruc- advantageously provided. 

tion function, e.g., IPDestroyProc, suitably allows an image Although the present invention has been described in 

processor to deallocate any internal storage allocated at accordance with the embodiments shown, one of ordinary 

initialization. It is appropriately called when the IPC 500 skill j Q mc art Wll i reC ognize that there could be variations 

containing the image processor 502 is being removed. In 45 lQ tfae embodimenl and thosc variations would be within the 

response to this call, the image processor 502 preferably spirit and scope of the present invention. Accordingly, many 

deallocates internal storage allocated in the initialization modifications may be made by one of ordinary skill without 

function call. departing from the spirit and scope of the present invention, 

Entry points to these seven functional routines for an the scope of which is defined by the following claims. What 

image processor 502, stored in an external data structure, 50 is claimed is: 

e.g., Functions, as well as the characteristics of the image What is claimed is: 

processor 502, are suitably stored in an internal data 1. A method for altering a linked series of image proces- 

structure, e.g., ImageProc, by the IPB 520. Once the image sors capable of manipulating digital image data, the method 

processors 502 arc defined through the seven functional comprising: 

routines and connected in an IPC 500, the IPB 520 suitably 55 providing a digital aptan dev ice with an image 

facilitates image processing operations by managing image processing chain; 

buffer I/O, and activation of each image processor 502 as , . . . . . . . 

u- . ■ « , ,f ; . t, . f coordinating, the image processing chain with an image 

soon as enough input data has been collected. The lnforma- . roccss jjj backplane* and 

tion for the image data processed is suitably stored in a data iprocessing acKp ane, an 

structure, e.g., Imagelnfo, including raw image size captured 60 modifying the image processing chain to form a new 

by a camera CCD, final processed output image size, bad processing chain through a plurality of functions 

pixel locations, etc. °f tDe " na g e processing backplane, including modify- 
ing through a function for deleting an image processor 

MODIFIABLE PARAMETER CONTROL OF from the image processing chain. 

IMAGE PROCESSOR IN AN IPC 65 2 . The method of claim 1 wherein the step of modifying 

In a preferred embodimenl, the IPB 520 further provides further comprises altering the image processing chain with 

routines to allow exchanges of parameter control settings by at least one externally provided image processor. 
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3. The method of claim 2 wherein the step of modifying 
further comprises inserting an image processor into the 
image processing chain. 

4. The method of claim 2 wherein the step of modifying 
further comprises replacing an image processor in the image 
processing chain. 

5. The method of claim 2 wherein the at least one 
externally provided image processor further comprises a 
linearization image processor. 

6. The method of claim 2 wherein the at least one 
externally provided image processor further comprises a bad 
pixel replacement image processor. 

7. The method of claim 2 wherein the at least one 
externally provided image processor further comprises a 
white balance image processor. 

8. The method of claim 2 wherein the at least one 
externally provided image processor further comprises an 
image reconstruction image processor. 

9. The method of claim 2 wherein the at least one 
externally provided image processor further comprises a 20 
color correction image processor. 

10. The method of claim 2 wherein the at least one 
externally provided image processor further comprises a 
color space conversion image processor. 

11. The method of claim 2 wherein the at least one 
externally provided image processor further comprises a 
sharpening image processor. 

12. The method of claim 2 wherein the at least one 
externally provided image processor further comprises a 
compression image processor. 

13. A system for manipulating digital image data, the 
system comprising: 

a digital image capture device, the digital image capture 
device capable of processing digital image data through 
two or more image processors; and 



25 



a central processing unit within the digital image capture 
device and capable of linking the two or more image 
processors to form an image processing chain, and 
coordinating the image processing chain with an image 
processing backplane, wherein the central processing 
unit facilitates modification of the image processing 
chain to form a new image processing chain, through a 
plurality of functions of the image processing 
backplane, including a function for deletion of an 
image processor from the image processing chain. 

14. The system of claim 13 wherein the central processing 
unit facilitates alteration of the image processing chain with 
at least one externally provided image processor. 

15. The system of claim 13 wherein the central processing 
unit facilitates insertion of an image processor into the 
image processing chain. 

16. The system of claim 13 wherein the central processing 
unit facilitates replacement of an image processor in the 
image processing chain. 

17. A computer readable medium containing program 
instructions for: 

providing a digital image capture device with an image 
processing chain; 

coordinating the image processing chain with an image 
processing backplane; and modifying the image pro- 
cessing chain to form a new image processing chain 
through a plurality of functions of the image processing 
backplane, including modifying through a function for 
deleting an image processor from the image processing 
chain. 
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